Dualisation, decision lists and identification of monotone discrete functions

Many data-analysis algorithms in machine learning, datamining and a variety of other disciplines essentially operate on discrete multi-attribute data sets. By means of discretisation or binarisation also numerical data sets can be successfully analysed. Therefore, in this paper we view/introduce the theory of (partially defined) discrete functions as an important theoretical tool for the analysis of multi-attribute data sets. In particular we study monotone (partially defined) discrete functions. Compared with the theory of Boolean functions relatively little is known about (partially defined) monotone discrete functions. It appears that decision lists are useful for the representation of monotone discrete functions. Since dualisation is an important tool in the theory of (monotone) Boolean functions, we study the interpretation and properties of the dual of a (monotone) binary or discrete function. We also introduce the dual of a pseudo-Boolean function. The results are used to investigate extensions of partially defined monotone discrete functions and the identification of monotone discrete functions. In particular we present a polynomial time algorithm for the identification of so-called stable discrete functions

Modular Decomposition of Boolean Functions

Modular decomposition is a thoroughly investigated topic in many areas such as switching theory, reliability theory, game theory and graph theory. Most appli- cations can be formulated in the framework of Boolean functions. In this paper we give a uni_ed treatment of modular decomposition of Boolean functions based on the idea of generalized Shannon decomposition. Furthermore, we discuss some new results on the complexity of modular decomposition. We propose an O(mn)- algorithm for the recognition of a modular set of a monotone Boolean function f with m prime implicants and n variables. Using this result we show that the the computation of the modular closure of a set can be done in time O(mn2). On the other hand, we prove that the recognition problem for general Boolean functions is coNP-complete

The Algorithmic Complexity of Modular Decomposition

Modular decomposition is a thoroughly investigated topic in many areas such as switching theory, reliability theory, game theory and graph theory. We propose an O(mn)-algorithm for the recognition of a modular set of a monotone Boolean function f with m prime implicants and n variables. Using this result we show that the computation of the modular closure of a set can be done in time O(mn2). On the other hand, we prove that the recognition problem for general Boolean func tions is NP-complete. Moreover, we introduce the so called generalized Shannon decomposition of a Boolean functions as an efficient tool for proving theorems on Boolean function decompositions

Mining frequent intemsets in memory-resident databases

Due to the present-day memory sizes, a memory-resident database has become a practical option. Consequently, new methods designed to mining in such databases are desirable. In the case of disk-resident databases, breadth-first search methods are commonly used. We propose a new algorithm, based upon depth-first search in a set-enumeration tree. For memory-resident databases, this method turns out to be superior to breadth-first search

Bankruptcy Prediction with Rough Sets

The bankruptcy prediction problem can be considered an or dinal classification problem. The classical theory of Rough Sets describes objects by discrete attributes, and does not take into account the order- ing of the attributes values. This paper proposes a modification of the Rough Set approach applicable to monotone datasets. We introduce re- spectively the concepts of monotone discernibility matrix and monotone (object) reduct. Furthermore, we use the theory of monotone discrete functions developed earlier by the first author to represent and to com- pute decision rules. In particular we use monotone extensions, decision lists and dualization to compute classification rules that cover the whole input space. The theory is applied to the bankruptcy prediction problem

Monotone Decision Trees and Noisy Data

The decision tree algorithm for monotone classification presented in [4, 10] requires strictly monotone data sets. This paper addresses the problem of noise due to violation of the monotonicity constraints and proposes a modification of the algorithm to handle noisy data. It also presents methods for controlling the size of the resulting trees while keeping the monotonicity property whether the data set is monotone or not

Induction of Ordinal Decision Trees

This paper focuses on the problem of monotone decision trees from the point of view of the multicriteria decision aid methodology (MCDA). By taking into account the preferences of the decision maker, an attempt is made to bring closer similar research within machine learning and MCDA. The paper addresses the question how to label the leaves of a tree in a way that guarantees the monotonicity of the resulting tree. Two approaches are proposed for that purpose - dynamic and static labeling which are also compared experimentally. The paper further considers the problem of splitting criteria in the con- text of monotone decision trees. Two criteria from the literature are com- pared experimentally - the entropy criterion and the number of con criterion - in an attempt to find out which one fits better the specifics of the monotone problems and which one better handles monotonicity noise

Simple improvements of a simple solution for inverting resolution

In this paper we address some simple improvements of the algorithm of Rouveirol and Puget [1989] for inverting resolution. Their approach is based on automatic change of representation called flattening and unflattening of clauses in a logic program. This enables a simple implementation of operators, such as Absorption, presented in Muggleton and Buntine [1988]. Unfortunately both the algorithms of MB and RP are incomplete. We analyze the reasons of the incompleteness of the RP algorithm and present an improved Absorption operator. It appears that flat tree epresentations of clauses and predicate calculus with equality provide an appropriate context for these matters

On the statistical mechanics of (un)constrained stochastic Hopfield and 'elastic' neural networks

Stochastic binary Hopfield models are viewed from the angle of statistical mechanics. After an analysis of the unconstrained model using mean field theory, a similar investigation is applied to a constrained model yielding comparable general explicit formulas of the free energy. Conditions are given for which some of the free energy expressions are Lyapunov functions of the corresponding differential equations. Both stochastic models appear to coincide with a specific continuous model. Physically, the models are related to spin and Potts glass models. Also, a `complementary' free energy function of both the unconstrained and the constrained model is derived. The analysis culminates in a very general framework for analyzing constrained and unconstrained Hopfield neural networks: the stationary points of the corresponding free energy appears to coincide exactly with the set of equilibrium conditions of the corresponding continuous Hopfield neural network. Moreover, the relationship with `elastic net' algorithms is analyzed: it is proved that this class of algorithms cannot be derived from the theory of statistical mechanics (as sometimes is supposed), but should be considered as a special `penalty method', namely as one with dynamical penalty weights. We mention some experimental results and discuss implications for the use of the various models in resolving constrained optimization problems

Monotone Decision Trees

EUR-FEW-CS-97-07 Title Monotone decision trees Author(s) R. Potharst J.C. Bioch T. Petter Abstract In many classification problems the domains of the attributes and the classes are linearly ordered. Often, classification must preserve this ordering: this is called monotone classification. Since the known decision tree methods generate non-monotone trees, these methods are not suitable for monotone classification problems. In this report we provide a number of order-preserving tree-generation algorithms for multi-attribute classification problems with k linearly ordered classes